Skip to content

feat: Voyage ai provider#166

Open
c58 wants to merge 6 commits intoyoanbernabeu:mainfrom
c58:voyage-ai-provider
Open

feat: Voyage ai provider#166
c58 wants to merge 6 commits intoyoanbernabeu:mainfrom
c58:voyage-ai-provider

Conversation

@c58
Copy link

@c58 c58 commented Feb 26, 2026

Description

Add Voyage AI as a new embedding provider for grepai. Voyage AI offers code-optimized embedding models (e.g., voyage-code-3) that produce 1024-dimensional vectors, providing a cloud-based alternative tuned specifically for code search use cases.

This PR implements:

  • Full VoyageAIEmbedder with batch embedding, adaptive rate limiting, token bucket throttling, and retry logic (including 429/5xx handling)
  • Integration into the embedder factory (embedder.NewFromConfig / embedder.NewFromWorkspaceConfig)
  • Interactive and non-interactive grepai init support (provider selection, endpoint/model defaults)
  • Configuration defaults (DefaultVoyageAIEmbeddingModel, DefaultVoyageAIEndpoint, dimensions=1024)
  • Correct dimension resolution in GetDimensions() for the voyageai provider
  • Support across all code paths: CLI search, watch, workspace search, workspace watch, and MCP server

Related Issue

Fixes #

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

How Has This Been Tested?

  • Unit tests
  • Integration tests
  • Manual testing

Test Configuration:

  • OS: Linux
  • Go version: (project default)
  • Embedding provider: voyageai (unit tests), ollama/openai (existing test suite)

Checklist

  • My code follows the project's code style
  • I have run golangci-lint run and fixed any issues
  • I have added tests that prove my fix/feature works
  • I have updated the documentation if needed
  • I have added an entry to CHANGELOG.md (if applicable)
  • My changes generate no new warnings
  • All new and existing tests pass

Screenshots (if applicable)

N/A

Additional Notes

  • The VOYAGE_API_KEY environment variable is used for authentication (falls back to config api_key field).
  • Voyage AI dimensions default to 1024 (native for voyage-code-3), distinct from OpenAI's 1536 and the local embedders' 768.
  • The embedder reuses the same batch/retry/rate-limiting infrastructure as the OpenAI embedder, with Voyage AI-specific request formatting (output_dimension field, input_type support).
  • This branch was merged with the factory refactor branch; the conflict resolution preserved both Voyage AI support and the centralized embedder.NewFromConfig factory pattern, avoiding code duplication across CLI commands and MCP server.

c58 added 5 commits February 26, 2026 17:16
* added voayge ai provider with customizable batching configuration per provider

* got through the PR checklist

* fixed init setting 768 dimension instead of using model default
@codecov
Copy link

codecov bot commented Feb 27, 2026

Codecov Report

❌ Patch coverage is 60.94891% with 107 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.33%. Comparing base (a322537) to head (6db14fc).
⚠️ Report is 97 commits behind head on main.

Files with missing lines Patch % Lines
embedder/voyageai.go 64.01% 67 Missing and 10 partials ⚠️
cli/init.go 5.88% 16 Missing ⚠️
embedder/batch.go 71.42% 5 Missing and 1 partial ⚠️
embedder/openai.go 0.00% 4 Missing ⚠️
config/config.go 75.00% 2 Missing ⚠️
embedder/factory.go 77.77% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #166       +/-   ##
===========================================
+ Coverage   27.16%   46.33%   +19.16%     
===========================================
  Files          32       71       +39     
  Lines        3711    13926    +10215     
===========================================
+ Hits         1008     6452     +5444     
- Misses       2620     6812     +4192     
- Partials       83      662      +579     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant